-
Notifications
You must be signed in to change notification settings - Fork 15.3k
[TargetLibraryInfo] Add libmvec support for risc-v #119844
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
@llvm/pr-subscribers-clang @llvm/pr-subscribers-clang-driver Author: Zhijin Zeng (zengdage) Changes
In VecFuncs.def, I add the LI_DEFINE_VECFUNC of LIBMVEC_RVV as follow: The The In the Patch is 57.62 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/119844.diff 17 Files Affected:
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index a020e00cd17392..1fdc1c66e9750e 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -5805,9 +5805,11 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
Triple.getArch() != llvm::Triple::x86_64)
D.Diag(diag::err_drv_unsupported_opt_for_target)
<< Name << Triple.getArchName();
- } else if (Name == "LIBMVEC-X86") {
+ } else if (Name == "LIBMVEC") {
if (Triple.getArch() != llvm::Triple::x86 &&
- Triple.getArch() != llvm::Triple::x86_64)
+ Triple.getArch() != llvm::Triple::x86_64 &&
+ Triple.getArch() != llvm::Triple::riscv32 &&
+ Triple.getArch() != llvm::Triple::riscv64)
D.Diag(diag::err_drv_unsupported_opt_for_target)
<< Name << Triple.getArchName();
} else if (Name == "SLEEF" || Name == "ArmPL") {
diff --git a/clang/lib/Driver/ToolChains/CommonArgs.cpp b/clang/lib/Driver/ToolChains/CommonArgs.cpp
index 0d851314a89539..2a4c298cfdeafb 100644
--- a/clang/lib/Driver/ToolChains/CommonArgs.cpp
+++ b/clang/lib/Driver/ToolChains/CommonArgs.cpp
@@ -909,7 +909,7 @@ void tools::addLTOOptions(const ToolChain &ToolChain, const ArgList &Args,
std::optional<StringRef> OptVal =
llvm::StringSwitch<std::optional<StringRef>>(ArgVecLib->getValue())
.Case("Accelerate", "Accelerate")
- .Case("LIBMVEC", "LIBMVEC-X86")
+ .Case("LIBMVEC", "LIBMVEC")
.Case("MASSV", "MASSV")
.Case("SVML", "SVML")
.Case("SLEEF", "sleefgnuabi")
diff --git a/clang/lib/Driver/ToolChains/Flang.cpp b/clang/lib/Driver/ToolChains/Flang.cpp
index c98fdbd157bac8..5d630d6e7777df 100644
--- a/clang/lib/Driver/ToolChains/Flang.cpp
+++ b/clang/lib/Driver/ToolChains/Flang.cpp
@@ -446,9 +446,11 @@ void Flang::addTargetOptions(const ArgList &Args,
Triple.getArch() != llvm::Triple::x86_64)
D.Diag(diag::err_drv_unsupported_opt_for_target)
<< Name << Triple.getArchName();
- } else if (Name == "LIBMVEC-X86") {
+ } else if (Name == "LIBMVEC") {
if (Triple.getArch() != llvm::Triple::x86 &&
- Triple.getArch() != llvm::Triple::x86_64)
+ Triple.getArch() != llvm::Triple::x86_64 &&
+ Triple.getArch() != llvm::Triple::riscv32 &&
+ Triple.getArch() != llvm::Triple::riscv64)
D.Diag(diag::err_drv_unsupported_opt_for_target)
<< Name << Triple.getArchName();
} else if (Name == "SLEEF" || Name == "ArmPL") {
diff --git a/clang/test/Driver/fveclib.c b/clang/test/Driver/fveclib.c
index 7d0985c4dd4f48..0b127cb540e176 100644
--- a/clang/test/Driver/fveclib.c
+++ b/clang/test/Driver/fveclib.c
@@ -5,6 +5,7 @@
// RUN: %clang -### -c -fveclib=Darwin_libsystem_m %s 2>&1 | FileCheck --check-prefix=CHECK-DARWIN_LIBSYSTEM_M %s
// RUN: %clang -### -c --target=aarch64 -fveclib=SLEEF %s 2>&1 | FileCheck --check-prefix=CHECK-SLEEF %s
// RUN: %clang -### -c --target=riscv64-unknown-linux-gnu -fveclib=SLEEF -march=rv64gcv %s 2>&1 | FileCheck -check-prefix CHECK-SLEEF-RISCV %s
+// RUN: %clang -### -c --target=riscv64-unknown-linux-gnu -fveclib=libmvec -march=rv64gcv %s 2>&1 | FileCheck -check-prefix CHECK-libmvec %s
// RUN: %clang -### -c --target=aarch64 -fveclib=ArmPL %s 2>&1 | FileCheck --check-prefix=CHECK-ARMPL %s
// RUN: not %clang -c -fveclib=something %s 2>&1 | FileCheck --check-prefix=CHECK-INVALID %s
@@ -21,7 +22,7 @@
// RUN: not %clang --target=x86 -c -fveclib=SLEEF %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
// RUN: not %clang --target=x86 -c -fveclib=ArmPL %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
-// RUN: not %clang --target=aarch64 -c -fveclib=LIBMVEC-X86 %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
+// RUN: not %clang --target=aarch64 -c -fveclib=LIBMVEC %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
// RUN: not %clang --target=aarch64 -c -fveclib=SVML %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
// CHECK-ERROR: unsupported option {{.*}} for target
@@ -38,7 +39,7 @@
/* Verify that the correct vector library is passed to LTO flags. */
// RUN: %clang -### --target=x86_64-unknown-linux-gnu -fveclib=LIBMVEC -flto %s 2>&1 | FileCheck --check-prefix=CHECK-LTO-LIBMVEC %s
-// CHECK-LTO-LIBMVEC: "-plugin-opt=-vector-library=LIBMVEC-X86"
+// CHECK-LTO-LIBMVEC: "-plugin-opt=-vector-library=LIBMVEC"
// RUN: %clang -### --target=powerpc64-unknown-linux-gnu -fveclib=MASSV -flto %s 2>&1 | FileCheck --check-prefix=CHECK-LTO-MASSV %s
// CHECK-LTO-MASSV: "-plugin-opt=-vector-library=MASSV"
@@ -52,6 +53,9 @@
// RUN: %clang -### --target=riscv64-unknown-linux-gnu -fveclib=SLEEF -flto -march=rv64gcv %s 2>&1 | FileCheck -check-prefix CHECK-LTO-SLEEF-RISCV %s
// CHECK-LTO-SLEEF-RISCV: "-plugin-opt=-vector-library=sleefgnuabi"
+// RUN: %clang -### --target=riscv64-unknown-linux-gnu -fveclib=LIBMVEC -flto -march=rv64gcv %s 2>&1 | FileCheck -check-prefix CHECK-LTO-LIBMVEC-RISCV %s
+// CHECK-LTO-LIBMVEC-RISCV: "-plugin-opt=-vector-library=LIBMVEC"
+
// RUN: %clang -### --target=aarch64-linux-gnu -fveclib=ArmPL -flto %s 2>&1 | FileCheck --check-prefix=CHECK-LTO-ARMPL %s
// CHECK-LTO-ARMPL: "-plugin-opt=-vector-library=ArmPL"
@@ -110,7 +114,7 @@
// CHECK-ENABLED-LAST: math errno enabled by '-ffp-model=strict' after it was implicitly disabled by '-fveclib=ArmPL', this may limit the utilization of the vector library [-Wmath-errno-enabled-with-veclib]
/* Verify no warning when math-errno is re-enabled for a different veclib (that does not imply -fno-math-errno). */
-// RUN: %clang -### --target=aarch64-linux-gnu -fveclib=ArmPL -fmath-errno -fveclib=LIBMVEC %s 2>&1 | FileCheck --check-prefix=CHECK-REPEAT-VECLIB %s
+// RUN: %clang -### --target=aarch64-linux-gnu -fveclib=ArmPL -fmath-errno -fveclib=LIBMVEC-X86 %s 2>&1 | FileCheck --check-prefix=CHECK-REPEAT-VECLIB %s
// CHECK-REPEAT-VECLIB-NOT: math errno enabled
/// Verify that vectorized routines library is being linked in.
diff --git a/llvm/include/llvm/Analysis/TargetLibraryInfo.h b/llvm/include/llvm/Analysis/TargetLibraryInfo.h
index f51d2bb9d50a21..b6bfcb865e20db 100644
--- a/llvm/include/llvm/Analysis/TargetLibraryInfo.h
+++ b/llvm/include/llvm/Analysis/TargetLibraryInfo.h
@@ -124,7 +124,7 @@ class TargetLibraryInfoImpl {
NoLibrary, // Don't use any vector library.
Accelerate, // Use Accelerate framework.
DarwinLibSystemM, // Use Darwin's libsystem_m.
- LIBMVEC_X86, // GLIBC Vector Math library.
+ LIBMVEC, // GLIBC Vector Math library.
MASSV, // IBM MASS vector library.
SVML, // Intel short vector math library.
SLEEFGNUABI, // SLEEF - SIMD Library for Evaluating Elementary Functions.
diff --git a/llvm/include/llvm/Analysis/VecFuncs.def b/llvm/include/llvm/Analysis/VecFuncs.def
index 251331b9f860c8..5e5a37dc39f687 100644
--- a/llvm/include/llvm/Analysis/VecFuncs.def
+++ b/llvm/include/llvm/Analysis/VecFuncs.def
@@ -236,6 +236,79 @@ TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGVdN4v_log", FIXED(4), "_ZGV_LLVM_N4v")
TLI_DEFINE_VECFUNC("llvm.log.f32", "_ZGVbN4v_logf", FIXED(4), "_ZGV_LLVM_N4v")
TLI_DEFINE_VECFUNC("llvm.log.f32", "_ZGVdN8v_logf", FIXED(8), "_ZGV_LLVM_N8v")
+#elif defined(TLI_DEFINE_LIBMVEC_RVV_VECFUNCS)
+// GLIBC Vector math Functions for RISC-V
+
+TLI_DEFINE_VECFUNC("sin", "_ZGV1Nxv_sin", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("sin", "_ZGV2Nxv_sin", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("sin", "_ZGV4Nxv_sin", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("sin", "_ZGV8Nxv_sin", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.sin.f64", "_ZGV1Nxv_sin", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.sin.f64", "_ZGV2Nxv_sin", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.sin.f64", "_ZGV4Nxv_sin", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.sin.f64", "_ZGV8Nxv_sin", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("cos", "_ZGV1Nxv_cos", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("cos", "_ZGV2Nxv_cos", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("cos", "_ZGV4Nxv_cos", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("cos", "_ZGV8Nxv_cos", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.cos.f64", "_ZGV1Nxv_cos", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.cos.f64", "_ZGV2Nxv_cos", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.cos.f64", "_ZGV4Nxv_cos", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.cos.f64", "_ZGV8Nxv_cos", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("tan", "_ZGV1Nxv_tan", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("tan", "_ZGV2Nxv_tan", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("tan", "_ZGV4Nxv_tan", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("tan", "_ZGV8Nxv_tan", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.tan.f64", "_ZGV1Nxv_tan", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.tan.f64", "_ZGV2Nxv_tan", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.tan.f64", "_ZGV4Nxv_tan", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.tan.f64", "_ZGV8Nxv_tan", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("pow", "_ZGV1Nxvv_pow", SCALABLE(1), "_ZGVr1Nxvv")
+TLI_DEFINE_VECFUNC("pow", "_ZGV2Nxvv_pow", SCALABLE(2), "_ZGVr2Nxvv")
+TLI_DEFINE_VECFUNC("pow", "_ZGV4Nxvv_pow", SCALABLE(4), "_ZGVr4Nxvv")
+TLI_DEFINE_VECFUNC("pow", "_ZGV8Nxvv_pow", SCALABLE(8), "_ZGVr8Nxvv")
+
+TLI_DEFINE_VECFUNC("llvm.pow.f64", "_ZGV1Nxvv_pow", SCALABLE(1), "_ZGVr1Nxvv")
+TLI_DEFINE_VECFUNC("llvm.pow.f64", "_ZGV2Nxvv_pow", SCALABLE(2), "_ZGVr2Nxvv")
+TLI_DEFINE_VECFUNC("llvm.pow.f64", "_ZGV4Nxvv_pow", SCALABLE(4), "_ZGVr4Nxvv")
+TLI_DEFINE_VECFUNC("llvm.pow.f64", "_ZGV8Nxvv_pow", SCALABLE(8), "_ZGVr8Nxvv")
+
+TLI_DEFINE_VECFUNC("exp", "_ZGV1Nxv_exp", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("exp", "_ZGV2Nxv_exp", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("exp", "_ZGV4Nxv_exp", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("exp", "_ZGV8Nxv_exp", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("expf", "_ZGV1Nxv_expf", SCALABLE(2), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("expf", "_ZGV2Nxv_expf", SCALABLE(4), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("expf", "_ZGV4Nxv_expf", SCALABLE(8), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("expf", "_ZGV8Nxv_expf", SCALABLE(16), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.exp.f64", "_ZGV1Nxv_exp", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f64", "_ZGV2Nxv_exp", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f64", "_ZGV4Nxv_exp", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f64", "_ZGV8Nxv_exp", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV1Nxv_expf", SCALABLE(2), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV2Nxv_expf", SCALABLE(4), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV4Nxv_expf", SCALABLE(8), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV8Nxv_expf", SCALABLE(16), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("log", "_ZGV1Nxv_log", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("log", "_ZGV2Nxv_log", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("log", "_ZGV4Nxv_log", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("log", "_ZGV8Nxv_log", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGV1Nxv_log", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGV2Nxv_log", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGV4Nxv_log", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGV8Nxv_log", SCALABLE(8), "_ZGVr8Nxv")
+
#elif defined(TLI_DEFINE_MASSV_VECFUNCS)
// IBM MASS library's vector Functions
diff --git a/llvm/include/llvm/IR/VFABIDemangler.h b/llvm/include/llvm/IR/VFABIDemangler.h
index de731cd7051c1b..283d42375fd4aa 100644
--- a/llvm/include/llvm/IR/VFABIDemangler.h
+++ b/llvm/include/llvm/IR/VFABIDemangler.h
@@ -49,6 +49,10 @@ enum class VFISAKind {
AVX, // x86 AVX
AVX2, // x86 AVX2
AVX512, // x86 AVX512
+ RVVM1, // RISC-V Vector Extension LMUL=1
+ RVVM2, // RISC-V Vector Extension LMUL=2
+ RVVM4, // RISC-V Vector Extension LMUL=4
+ RVVM8, // RISC-V Vector Extension LMUL=8
LLVM, // LLVM internal ISA for functions that are not
// attached to an existing ABI via name mangling.
Unknown // Unknown ISA
diff --git a/llvm/lib/Analysis/TargetLibraryInfo.cpp b/llvm/lib/Analysis/TargetLibraryInfo.cpp
index 8557901192e406..0135c7a9b9ce7d 100644
--- a/llvm/lib/Analysis/TargetLibraryInfo.cpp
+++ b/llvm/lib/Analysis/TargetLibraryInfo.cpp
@@ -29,7 +29,7 @@ static cl::opt<TargetLibraryInfoImpl::VectorLibrary> ClVectorLibrary(
"Accelerate framework"),
clEnumValN(TargetLibraryInfoImpl::DarwinLibSystemM,
"Darwin_libsystem_m", "Darwin libsystem_m"),
- clEnumValN(TargetLibraryInfoImpl::LIBMVEC_X86, "LIBMVEC-X86",
+ clEnumValN(TargetLibraryInfoImpl::LIBMVEC, "LIBMVEC",
"GLIBC Vector Math library"),
clEnumValN(TargetLibraryInfoImpl::MASSV, "MASSV",
"IBM MASS vector library"),
@@ -1291,6 +1291,12 @@ static const VecDesc VecFuncs_LIBMVEC_X86[] = {
#undef TLI_DEFINE_LIBMVEC_X86_VECFUNCS
};
+static const VecDesc VecFuncs_LIBMVEC_RVV[] = {
+#define TLI_DEFINE_LIBMVEC_RVV_VECFUNCS
+#include "llvm/Analysis/VecFuncs.def"
+#undef TLI_DEFINE_LIBMVEC_RVV_VECFUNCS
+};
+
static const VecDesc VecFuncs_MASSV[] = {
#define TLI_DEFINE_MASSV_VECFUNCS
#include "llvm/Analysis/VecFuncs.def"
@@ -1360,8 +1366,19 @@ void TargetLibraryInfoImpl::addVectorizableFunctionsFromVecLib(
addVectorizableFunctions(VecFuncs_DarwinLibSystemM);
break;
}
- case LIBMVEC_X86: {
- addVectorizableFunctions(VecFuncs_LIBMVEC_X86);
+ case LIBMVEC: {
+ switch (TargetTriple.getArch()) {
+ default:
+ break;
+ case llvm::Triple::x86:
+ case llvm::Triple::x86_64:
+ addVectorizableFunctions(VecFuncs_LIBMVEC_X86);
+ break;
+ case llvm::Triple::riscv64:
+ case llvm::Triple::riscv32:
+ addVectorizableFunctions(VecFuncs_LIBMVEC_RVV);
+ break;
+ }
break;
}
case MASSV: {
diff --git a/llvm/lib/Frontend/Driver/CodeGenOptions.cpp b/llvm/lib/Frontend/Driver/CodeGenOptions.cpp
index 2d74a91f62dc07..e368a15a09bf34 100644
--- a/llvm/lib/Frontend/Driver/CodeGenOptions.cpp
+++ b/llvm/lib/Frontend/Driver/CodeGenOptions.cpp
@@ -23,7 +23,7 @@ TargetLibraryInfoImpl *createTLII(llvm::Triple &TargetTriple,
TargetTriple);
break;
case VectorLibrary::LIBMVEC:
- TLII->addVectorizableFunctionsFromVecLib(TargetLibraryInfoImpl::LIBMVEC_X86,
+ TLII->addVectorizableFunctionsFromVecLib(TargetLibraryInfoImpl::LIBMVEC,
TargetTriple);
break;
case VectorLibrary::MASSV:
diff --git a/llvm/lib/IR/VFABIDemangler.cpp b/llvm/lib/IR/VFABIDemangler.cpp
index 897583084bf38c..07befb7b89b978 100644
--- a/llvm/lib/IR/VFABIDemangler.cpp
+++ b/llvm/lib/IR/VFABIDemangler.cpp
@@ -38,11 +38,19 @@ static ParseRet tryParseISA(StringRef &MangledName, VFISAKind &ISA) {
if (MangledName.consume_front(VFABI::_LLVM_)) {
ISA = VFISAKind::LLVM;
+ } else if (MangledName.consume_front("r")) {
+ ISA = StringSwitch<VFISAKind>(MangledName.take_front(1))
+ .Case("1", VFISAKind::RVVM1)
+ .Case("2", VFISAKind::RVVM2)
+ .Case("4", VFISAKind::RVVM4)
+ .Case("8", VFISAKind::RVVM8)
+ .Default(VFISAKind::RVV);
+ if (ISA != VFISAKind::RVV)
+ MangledName = MangledName.drop_front(1);
} else {
ISA = StringSwitch<VFISAKind>(MangledName.take_front(1))
.Case("n", VFISAKind::AdvancedSIMD)
.Case("s", VFISAKind::SVE)
- .Case("r", VFISAKind::RVV)
.Case("b", VFISAKind::SSE)
.Case("c", VFISAKind::AVX)
.Case("d", VFISAKind::AVX2)
@@ -79,8 +87,10 @@ static ParseRet tryParseMask(StringRef &MangledName, bool &IsMasked) {
static ParseRet tryParseVLEN(StringRef &ParseString, VFISAKind ISA,
std::pair<unsigned, bool> &ParsedVF) {
if (ParseString.consume_front("x")) {
- // SVE is the only scalable ISA currently supported.
- if (ISA != VFISAKind::SVE && ISA != VFISAKind::RVV) {
+ // SVE/RVV is the only two scalable ISAs currently supported.
+ if (ISA != VFISAKind::SVE && ISA != VFISAKind::RVV &&
+ ISA != VFISAKind::RVVM1 && ISA != VFISAKind::RVVM2 &&
+ ISA != VFISAKind::RVVM4 && ISA != VFISAKind::RVVM8) {
LLVM_DEBUG(dbgs() << "Vector function variant declared with scalable VF "
<< "but ISA supported for SVE and RVV only\n");
return ParseRet::Error;
@@ -302,17 +312,52 @@ static ParseRet tryParseAlign(StringRef &ParseString, Align &Alignment) {
// the number of elements of the given type which would fit in such a vector.
static std::optional<ElementCount> getElementCountForTy(const VFISAKind ISA,
const Type *Ty) {
- assert((ISA == VFISAKind::SVE || ISA == VFISAKind::RVV) &&
+ // Only AArch64 SVE and RVV are supported at present.
+ assert((ISA == VFISAKind::SVE || ISA == VFISAKind::RVV ||
+ ISA == VFISAKind::RVVM1 || ISA == VFISAKind::RVVM2 ||
+ ISA == VFISAKind::RVVM4 || ISA == VFISAKind::RVVM8) &&
"Scalable VF decoding only implemented for SVE and RVV\n");
- if (Ty->isIntegerTy(64) || Ty->isDoubleTy() || Ty->isPointerTy())
- return ElementCount::getScalable(2);
- if (Ty->isIntegerTy(32) || Ty->isFloatTy())
- return ElementCount::getScalable(4);
- if (Ty->isIntegerTy(16) || Ty->is16bitFPTy())
- return ElementCount::getScalable(8);
- if (Ty->isIntegerTy(8))
- return ElementCount::getScalable(16);
+ if (ISA == VFISAKind::SVE || ISA == VFISAKind::RVV) {
+ if (Ty->isIntegerTy(64) || Ty->isDoubleTy() || Ty->isPointerTy())
+ return ElementCount::getScalable(2);
+ if (Ty->isIntegerTy(32) || Ty->isFloatTy())
+ return ElementCount::getScalable(4);
+ if (Ty->isIntegerTy(16) || Ty->is16bitFPTy())
+ return ElementCount::getScalable(8);
+ if (Ty->isIntegerTy(8))
+ return ElementCount::getScalable(16);
+ } else if (ISA == VFISAKind::RVVM1 || ISA == VFISAKind::RVVM2 ||
+ ISA == VFISAKind::RVVM4 || ISA == VFISAKind::RVVM8) {
+ // Because 'vscale = VLENB/8', so the ElementCount should be
+ // 'vscale x (LMUL * 64 / sizeof(Type))'.
+ unsigned Number = 1;
+ unsigned LMUL = 1;
+ unsigned ElemCount;
+
+ // TODO: need to distingush rv32 and rv64.
+ if (Ty->isPointerTy())
+ return std::nullopt;
+
+ if (Ty->isIntegerTy(64) || Ty->isDoubleTy())
+ Number = 1;
+ if (Ty->isIntegerTy(32) || Ty->isFloatTy())
+ Number = 2;
+ if (Ty->isIntegerTy(16) || Ty->is16bitFPTy())
+ Number = 4;
+ if (Ty->isIntegerTy(8))
+ Number = 8;
+
+ if (ISA == VFISAKind::RVVM2)
+ LMUL = 2;
+ else if (ISA == VFISAKind::RVVM4)
+ LMUL = 4;
+ else if (ISA == VFISAKind::RVVM8)
+ LMUL = 8;
+
+ ElemCount = LMUL * Number;
+ return ElementCount::getScalable(ElemCount);
+ }
return std::nullopt;
}
diff --git a/llvm/lib/Transforms/Utils/InjectTLIMappings.cpp b/llvm/lib/Transforms/Utils/InjectTLIMappings.cpp
index 92b5d444aac340..8456e4294083bf 100644
--- a/llvm/lib/Transforms/Utils/InjectTLIMappings.cpp
+++ b/llvm/lib/Transforms/Utils/InjectTLIMappings.cpp
@@ -109,11 +109,11 @@ static void addMappingsFromTLI(const TargetLibraryInfo &TLI, CallInst &CI) {
TLI.getWidestVF(ScalarName, WidestFixedVF, WidestScalableVF);
for (bool Predicated : {false, true}) {
- for (ElementCount VF = ElementCount::getFixed(2);
+ for (ElementCount VF = ElementCount::getFixed(1);
ElementCount::isKnownLE(VF, WidestFixedVF); VF *= 2)
AddVariantDecl(VF, Predicated);
- for (ElementCount VF = ElementCount::getScalable(2);
+ for (ElementCount VF = ElementCount::getScalable(1);
ElementCount::isKnownLE(VF, WidestScalableVF); VF *= 2)
AddVariantDecl(VF, Predicated);
}
diff --git a/llvm/test/CodeGen/Generic/replace-intrinsics-with-veclib.ll b/llvm/test/CodeGen/Generic/replace-intrinsics-with-veclib.ll
index fde6cb788b46f9..1bd929f836f9ed 100644
--- a/ll...
[truncated]
|
|
@llvm/pr-subscribers-llvm-ir Author: Zhijin Zeng (zengdage) Changes
In VecFuncs.def, I add the LI_DEFINE_VECFUNC of LIBMVEC_RVV as follow: The The In the Patch is 57.62 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/119844.diff 17 Files Affected:
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index a020e00cd17392..1fdc1c66e9750e 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -5805,9 +5805,11 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
Triple.getArch() != llvm::Triple::x86_64)
D.Diag(diag::err_drv_unsupported_opt_for_target)
<< Name << Triple.getArchName();
- } else if (Name == "LIBMVEC-X86") {
+ } else if (Name == "LIBMVEC") {
if (Triple.getArch() != llvm::Triple::x86 &&
- Triple.getArch() != llvm::Triple::x86_64)
+ Triple.getArch() != llvm::Triple::x86_64 &&
+ Triple.getArch() != llvm::Triple::riscv32 &&
+ Triple.getArch() != llvm::Triple::riscv64)
D.Diag(diag::err_drv_unsupported_opt_for_target)
<< Name << Triple.getArchName();
} else if (Name == "SLEEF" || Name == "ArmPL") {
diff --git a/clang/lib/Driver/ToolChains/CommonArgs.cpp b/clang/lib/Driver/ToolChains/CommonArgs.cpp
index 0d851314a89539..2a4c298cfdeafb 100644
--- a/clang/lib/Driver/ToolChains/CommonArgs.cpp
+++ b/clang/lib/Driver/ToolChains/CommonArgs.cpp
@@ -909,7 +909,7 @@ void tools::addLTOOptions(const ToolChain &ToolChain, const ArgList &Args,
std::optional<StringRef> OptVal =
llvm::StringSwitch<std::optional<StringRef>>(ArgVecLib->getValue())
.Case("Accelerate", "Accelerate")
- .Case("LIBMVEC", "LIBMVEC-X86")
+ .Case("LIBMVEC", "LIBMVEC")
.Case("MASSV", "MASSV")
.Case("SVML", "SVML")
.Case("SLEEF", "sleefgnuabi")
diff --git a/clang/lib/Driver/ToolChains/Flang.cpp b/clang/lib/Driver/ToolChains/Flang.cpp
index c98fdbd157bac8..5d630d6e7777df 100644
--- a/clang/lib/Driver/ToolChains/Flang.cpp
+++ b/clang/lib/Driver/ToolChains/Flang.cpp
@@ -446,9 +446,11 @@ void Flang::addTargetOptions(const ArgList &Args,
Triple.getArch() != llvm::Triple::x86_64)
D.Diag(diag::err_drv_unsupported_opt_for_target)
<< Name << Triple.getArchName();
- } else if (Name == "LIBMVEC-X86") {
+ } else if (Name == "LIBMVEC") {
if (Triple.getArch() != llvm::Triple::x86 &&
- Triple.getArch() != llvm::Triple::x86_64)
+ Triple.getArch() != llvm::Triple::x86_64 &&
+ Triple.getArch() != llvm::Triple::riscv32 &&
+ Triple.getArch() != llvm::Triple::riscv64)
D.Diag(diag::err_drv_unsupported_opt_for_target)
<< Name << Triple.getArchName();
} else if (Name == "SLEEF" || Name == "ArmPL") {
diff --git a/clang/test/Driver/fveclib.c b/clang/test/Driver/fveclib.c
index 7d0985c4dd4f48..0b127cb540e176 100644
--- a/clang/test/Driver/fveclib.c
+++ b/clang/test/Driver/fveclib.c
@@ -5,6 +5,7 @@
// RUN: %clang -### -c -fveclib=Darwin_libsystem_m %s 2>&1 | FileCheck --check-prefix=CHECK-DARWIN_LIBSYSTEM_M %s
// RUN: %clang -### -c --target=aarch64 -fveclib=SLEEF %s 2>&1 | FileCheck --check-prefix=CHECK-SLEEF %s
// RUN: %clang -### -c --target=riscv64-unknown-linux-gnu -fveclib=SLEEF -march=rv64gcv %s 2>&1 | FileCheck -check-prefix CHECK-SLEEF-RISCV %s
+// RUN: %clang -### -c --target=riscv64-unknown-linux-gnu -fveclib=libmvec -march=rv64gcv %s 2>&1 | FileCheck -check-prefix CHECK-libmvec %s
// RUN: %clang -### -c --target=aarch64 -fveclib=ArmPL %s 2>&1 | FileCheck --check-prefix=CHECK-ARMPL %s
// RUN: not %clang -c -fveclib=something %s 2>&1 | FileCheck --check-prefix=CHECK-INVALID %s
@@ -21,7 +22,7 @@
// RUN: not %clang --target=x86 -c -fveclib=SLEEF %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
// RUN: not %clang --target=x86 -c -fveclib=ArmPL %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
-// RUN: not %clang --target=aarch64 -c -fveclib=LIBMVEC-X86 %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
+// RUN: not %clang --target=aarch64 -c -fveclib=LIBMVEC %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
// RUN: not %clang --target=aarch64 -c -fveclib=SVML %s 2>&1 | FileCheck --check-prefix=CHECK-ERROR %s
// CHECK-ERROR: unsupported option {{.*}} for target
@@ -38,7 +39,7 @@
/* Verify that the correct vector library is passed to LTO flags. */
// RUN: %clang -### --target=x86_64-unknown-linux-gnu -fveclib=LIBMVEC -flto %s 2>&1 | FileCheck --check-prefix=CHECK-LTO-LIBMVEC %s
-// CHECK-LTO-LIBMVEC: "-plugin-opt=-vector-library=LIBMVEC-X86"
+// CHECK-LTO-LIBMVEC: "-plugin-opt=-vector-library=LIBMVEC"
// RUN: %clang -### --target=powerpc64-unknown-linux-gnu -fveclib=MASSV -flto %s 2>&1 | FileCheck --check-prefix=CHECK-LTO-MASSV %s
// CHECK-LTO-MASSV: "-plugin-opt=-vector-library=MASSV"
@@ -52,6 +53,9 @@
// RUN: %clang -### --target=riscv64-unknown-linux-gnu -fveclib=SLEEF -flto -march=rv64gcv %s 2>&1 | FileCheck -check-prefix CHECK-LTO-SLEEF-RISCV %s
// CHECK-LTO-SLEEF-RISCV: "-plugin-opt=-vector-library=sleefgnuabi"
+// RUN: %clang -### --target=riscv64-unknown-linux-gnu -fveclib=LIBMVEC -flto -march=rv64gcv %s 2>&1 | FileCheck -check-prefix CHECK-LTO-LIBMVEC-RISCV %s
+// CHECK-LTO-LIBMVEC-RISCV: "-plugin-opt=-vector-library=LIBMVEC"
+
// RUN: %clang -### --target=aarch64-linux-gnu -fveclib=ArmPL -flto %s 2>&1 | FileCheck --check-prefix=CHECK-LTO-ARMPL %s
// CHECK-LTO-ARMPL: "-plugin-opt=-vector-library=ArmPL"
@@ -110,7 +114,7 @@
// CHECK-ENABLED-LAST: math errno enabled by '-ffp-model=strict' after it was implicitly disabled by '-fveclib=ArmPL', this may limit the utilization of the vector library [-Wmath-errno-enabled-with-veclib]
/* Verify no warning when math-errno is re-enabled for a different veclib (that does not imply -fno-math-errno). */
-// RUN: %clang -### --target=aarch64-linux-gnu -fveclib=ArmPL -fmath-errno -fveclib=LIBMVEC %s 2>&1 | FileCheck --check-prefix=CHECK-REPEAT-VECLIB %s
+// RUN: %clang -### --target=aarch64-linux-gnu -fveclib=ArmPL -fmath-errno -fveclib=LIBMVEC-X86 %s 2>&1 | FileCheck --check-prefix=CHECK-REPEAT-VECLIB %s
// CHECK-REPEAT-VECLIB-NOT: math errno enabled
/// Verify that vectorized routines library is being linked in.
diff --git a/llvm/include/llvm/Analysis/TargetLibraryInfo.h b/llvm/include/llvm/Analysis/TargetLibraryInfo.h
index f51d2bb9d50a21..b6bfcb865e20db 100644
--- a/llvm/include/llvm/Analysis/TargetLibraryInfo.h
+++ b/llvm/include/llvm/Analysis/TargetLibraryInfo.h
@@ -124,7 +124,7 @@ class TargetLibraryInfoImpl {
NoLibrary, // Don't use any vector library.
Accelerate, // Use Accelerate framework.
DarwinLibSystemM, // Use Darwin's libsystem_m.
- LIBMVEC_X86, // GLIBC Vector Math library.
+ LIBMVEC, // GLIBC Vector Math library.
MASSV, // IBM MASS vector library.
SVML, // Intel short vector math library.
SLEEFGNUABI, // SLEEF - SIMD Library for Evaluating Elementary Functions.
diff --git a/llvm/include/llvm/Analysis/VecFuncs.def b/llvm/include/llvm/Analysis/VecFuncs.def
index 251331b9f860c8..5e5a37dc39f687 100644
--- a/llvm/include/llvm/Analysis/VecFuncs.def
+++ b/llvm/include/llvm/Analysis/VecFuncs.def
@@ -236,6 +236,79 @@ TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGVdN4v_log", FIXED(4), "_ZGV_LLVM_N4v")
TLI_DEFINE_VECFUNC("llvm.log.f32", "_ZGVbN4v_logf", FIXED(4), "_ZGV_LLVM_N4v")
TLI_DEFINE_VECFUNC("llvm.log.f32", "_ZGVdN8v_logf", FIXED(8), "_ZGV_LLVM_N8v")
+#elif defined(TLI_DEFINE_LIBMVEC_RVV_VECFUNCS)
+// GLIBC Vector math Functions for RISC-V
+
+TLI_DEFINE_VECFUNC("sin", "_ZGV1Nxv_sin", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("sin", "_ZGV2Nxv_sin", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("sin", "_ZGV4Nxv_sin", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("sin", "_ZGV8Nxv_sin", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.sin.f64", "_ZGV1Nxv_sin", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.sin.f64", "_ZGV2Nxv_sin", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.sin.f64", "_ZGV4Nxv_sin", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.sin.f64", "_ZGV8Nxv_sin", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("cos", "_ZGV1Nxv_cos", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("cos", "_ZGV2Nxv_cos", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("cos", "_ZGV4Nxv_cos", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("cos", "_ZGV8Nxv_cos", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.cos.f64", "_ZGV1Nxv_cos", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.cos.f64", "_ZGV2Nxv_cos", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.cos.f64", "_ZGV4Nxv_cos", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.cos.f64", "_ZGV8Nxv_cos", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("tan", "_ZGV1Nxv_tan", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("tan", "_ZGV2Nxv_tan", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("tan", "_ZGV4Nxv_tan", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("tan", "_ZGV8Nxv_tan", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.tan.f64", "_ZGV1Nxv_tan", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.tan.f64", "_ZGV2Nxv_tan", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.tan.f64", "_ZGV4Nxv_tan", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.tan.f64", "_ZGV8Nxv_tan", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("pow", "_ZGV1Nxvv_pow", SCALABLE(1), "_ZGVr1Nxvv")
+TLI_DEFINE_VECFUNC("pow", "_ZGV2Nxvv_pow", SCALABLE(2), "_ZGVr2Nxvv")
+TLI_DEFINE_VECFUNC("pow", "_ZGV4Nxvv_pow", SCALABLE(4), "_ZGVr4Nxvv")
+TLI_DEFINE_VECFUNC("pow", "_ZGV8Nxvv_pow", SCALABLE(8), "_ZGVr8Nxvv")
+
+TLI_DEFINE_VECFUNC("llvm.pow.f64", "_ZGV1Nxvv_pow", SCALABLE(1), "_ZGVr1Nxvv")
+TLI_DEFINE_VECFUNC("llvm.pow.f64", "_ZGV2Nxvv_pow", SCALABLE(2), "_ZGVr2Nxvv")
+TLI_DEFINE_VECFUNC("llvm.pow.f64", "_ZGV4Nxvv_pow", SCALABLE(4), "_ZGVr4Nxvv")
+TLI_DEFINE_VECFUNC("llvm.pow.f64", "_ZGV8Nxvv_pow", SCALABLE(8), "_ZGVr8Nxvv")
+
+TLI_DEFINE_VECFUNC("exp", "_ZGV1Nxv_exp", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("exp", "_ZGV2Nxv_exp", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("exp", "_ZGV4Nxv_exp", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("exp", "_ZGV8Nxv_exp", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("expf", "_ZGV1Nxv_expf", SCALABLE(2), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("expf", "_ZGV2Nxv_expf", SCALABLE(4), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("expf", "_ZGV4Nxv_expf", SCALABLE(8), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("expf", "_ZGV8Nxv_expf", SCALABLE(16), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.exp.f64", "_ZGV1Nxv_exp", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f64", "_ZGV2Nxv_exp", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f64", "_ZGV4Nxv_exp", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f64", "_ZGV8Nxv_exp", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV1Nxv_expf", SCALABLE(2), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV2Nxv_expf", SCALABLE(4), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV4Nxv_expf", SCALABLE(8), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV8Nxv_expf", SCALABLE(16), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("log", "_ZGV1Nxv_log", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("log", "_ZGV2Nxv_log", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("log", "_ZGV4Nxv_log", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("log", "_ZGV8Nxv_log", SCALABLE(8), "_ZGVr8Nxv")
+
+TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGV1Nxv_log", SCALABLE(1), "_ZGVr1Nxv")
+TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGV2Nxv_log", SCALABLE(2), "_ZGVr2Nxv")
+TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGV4Nxv_log", SCALABLE(4), "_ZGVr4Nxv")
+TLI_DEFINE_VECFUNC("llvm.log.f64", "_ZGV8Nxv_log", SCALABLE(8), "_ZGVr8Nxv")
+
#elif defined(TLI_DEFINE_MASSV_VECFUNCS)
// IBM MASS library's vector Functions
diff --git a/llvm/include/llvm/IR/VFABIDemangler.h b/llvm/include/llvm/IR/VFABIDemangler.h
index de731cd7051c1b..283d42375fd4aa 100644
--- a/llvm/include/llvm/IR/VFABIDemangler.h
+++ b/llvm/include/llvm/IR/VFABIDemangler.h
@@ -49,6 +49,10 @@ enum class VFISAKind {
AVX, // x86 AVX
AVX2, // x86 AVX2
AVX512, // x86 AVX512
+ RVVM1, // RISC-V Vector Extension LMUL=1
+ RVVM2, // RISC-V Vector Extension LMUL=2
+ RVVM4, // RISC-V Vector Extension LMUL=4
+ RVVM8, // RISC-V Vector Extension LMUL=8
LLVM, // LLVM internal ISA for functions that are not
// attached to an existing ABI via name mangling.
Unknown // Unknown ISA
diff --git a/llvm/lib/Analysis/TargetLibraryInfo.cpp b/llvm/lib/Analysis/TargetLibraryInfo.cpp
index 8557901192e406..0135c7a9b9ce7d 100644
--- a/llvm/lib/Analysis/TargetLibraryInfo.cpp
+++ b/llvm/lib/Analysis/TargetLibraryInfo.cpp
@@ -29,7 +29,7 @@ static cl::opt<TargetLibraryInfoImpl::VectorLibrary> ClVectorLibrary(
"Accelerate framework"),
clEnumValN(TargetLibraryInfoImpl::DarwinLibSystemM,
"Darwin_libsystem_m", "Darwin libsystem_m"),
- clEnumValN(TargetLibraryInfoImpl::LIBMVEC_X86, "LIBMVEC-X86",
+ clEnumValN(TargetLibraryInfoImpl::LIBMVEC, "LIBMVEC",
"GLIBC Vector Math library"),
clEnumValN(TargetLibraryInfoImpl::MASSV, "MASSV",
"IBM MASS vector library"),
@@ -1291,6 +1291,12 @@ static const VecDesc VecFuncs_LIBMVEC_X86[] = {
#undef TLI_DEFINE_LIBMVEC_X86_VECFUNCS
};
+static const VecDesc VecFuncs_LIBMVEC_RVV[] = {
+#define TLI_DEFINE_LIBMVEC_RVV_VECFUNCS
+#include "llvm/Analysis/VecFuncs.def"
+#undef TLI_DEFINE_LIBMVEC_RVV_VECFUNCS
+};
+
static const VecDesc VecFuncs_MASSV[] = {
#define TLI_DEFINE_MASSV_VECFUNCS
#include "llvm/Analysis/VecFuncs.def"
@@ -1360,8 +1366,19 @@ void TargetLibraryInfoImpl::addVectorizableFunctionsFromVecLib(
addVectorizableFunctions(VecFuncs_DarwinLibSystemM);
break;
}
- case LIBMVEC_X86: {
- addVectorizableFunctions(VecFuncs_LIBMVEC_X86);
+ case LIBMVEC: {
+ switch (TargetTriple.getArch()) {
+ default:
+ break;
+ case llvm::Triple::x86:
+ case llvm::Triple::x86_64:
+ addVectorizableFunctions(VecFuncs_LIBMVEC_X86);
+ break;
+ case llvm::Triple::riscv64:
+ case llvm::Triple::riscv32:
+ addVectorizableFunctions(VecFuncs_LIBMVEC_RVV);
+ break;
+ }
break;
}
case MASSV: {
diff --git a/llvm/lib/Frontend/Driver/CodeGenOptions.cpp b/llvm/lib/Frontend/Driver/CodeGenOptions.cpp
index 2d74a91f62dc07..e368a15a09bf34 100644
--- a/llvm/lib/Frontend/Driver/CodeGenOptions.cpp
+++ b/llvm/lib/Frontend/Driver/CodeGenOptions.cpp
@@ -23,7 +23,7 @@ TargetLibraryInfoImpl *createTLII(llvm::Triple &TargetTriple,
TargetTriple);
break;
case VectorLibrary::LIBMVEC:
- TLII->addVectorizableFunctionsFromVecLib(TargetLibraryInfoImpl::LIBMVEC_X86,
+ TLII->addVectorizableFunctionsFromVecLib(TargetLibraryInfoImpl::LIBMVEC,
TargetTriple);
break;
case VectorLibrary::MASSV:
diff --git a/llvm/lib/IR/VFABIDemangler.cpp b/llvm/lib/IR/VFABIDemangler.cpp
index 897583084bf38c..07befb7b89b978 100644
--- a/llvm/lib/IR/VFABIDemangler.cpp
+++ b/llvm/lib/IR/VFABIDemangler.cpp
@@ -38,11 +38,19 @@ static ParseRet tryParseISA(StringRef &MangledName, VFISAKind &ISA) {
if (MangledName.consume_front(VFABI::_LLVM_)) {
ISA = VFISAKind::LLVM;
+ } else if (MangledName.consume_front("r")) {
+ ISA = StringSwitch<VFISAKind>(MangledName.take_front(1))
+ .Case("1", VFISAKind::RVVM1)
+ .Case("2", VFISAKind::RVVM2)
+ .Case("4", VFISAKind::RVVM4)
+ .Case("8", VFISAKind::RVVM8)
+ .Default(VFISAKind::RVV);
+ if (ISA != VFISAKind::RVV)
+ MangledName = MangledName.drop_front(1);
} else {
ISA = StringSwitch<VFISAKind>(MangledName.take_front(1))
.Case("n", VFISAKind::AdvancedSIMD)
.Case("s", VFISAKind::SVE)
- .Case("r", VFISAKind::RVV)
.Case("b", VFISAKind::SSE)
.Case("c", VFISAKind::AVX)
.Case("d", VFISAKind::AVX2)
@@ -79,8 +87,10 @@ static ParseRet tryParseMask(StringRef &MangledName, bool &IsMasked) {
static ParseRet tryParseVLEN(StringRef &ParseString, VFISAKind ISA,
std::pair<unsigned, bool> &ParsedVF) {
if (ParseString.consume_front("x")) {
- // SVE is the only scalable ISA currently supported.
- if (ISA != VFISAKind::SVE && ISA != VFISAKind::RVV) {
+ // SVE/RVV is the only two scalable ISAs currently supported.
+ if (ISA != VFISAKind::SVE && ISA != VFISAKind::RVV &&
+ ISA != VFISAKind::RVVM1 && ISA != VFISAKind::RVVM2 &&
+ ISA != VFISAKind::RVVM4 && ISA != VFISAKind::RVVM8) {
LLVM_DEBUG(dbgs() << "Vector function variant declared with scalable VF "
<< "but ISA supported for SVE and RVV only\n");
return ParseRet::Error;
@@ -302,17 +312,52 @@ static ParseRet tryParseAlign(StringRef &ParseString, Align &Alignment) {
// the number of elements of the given type which would fit in such a vector.
static std::optional<ElementCount> getElementCountForTy(const VFISAKind ISA,
const Type *Ty) {
- assert((ISA == VFISAKind::SVE || ISA == VFISAKind::RVV) &&
+ // Only AArch64 SVE and RVV are supported at present.
+ assert((ISA == VFISAKind::SVE || ISA == VFISAKind::RVV ||
+ ISA == VFISAKind::RVVM1 || ISA == VFISAKind::RVVM2 ||
+ ISA == VFISAKind::RVVM4 || ISA == VFISAKind::RVVM8) &&
"Scalable VF decoding only implemented for SVE and RVV\n");
- if (Ty->isIntegerTy(64) || Ty->isDoubleTy() || Ty->isPointerTy())
- return ElementCount::getScalable(2);
- if (Ty->isIntegerTy(32) || Ty->isFloatTy())
- return ElementCount::getScalable(4);
- if (Ty->isIntegerTy(16) || Ty->is16bitFPTy())
- return ElementCount::getScalable(8);
- if (Ty->isIntegerTy(8))
- return ElementCount::getScalable(16);
+ if (ISA == VFISAKind::SVE || ISA == VFISAKind::RVV) {
+ if (Ty->isIntegerTy(64) || Ty->isDoubleTy() || Ty->isPointerTy())
+ return ElementCount::getScalable(2);
+ if (Ty->isIntegerTy(32) || Ty->isFloatTy())
+ return ElementCount::getScalable(4);
+ if (Ty->isIntegerTy(16) || Ty->is16bitFPTy())
+ return ElementCount::getScalable(8);
+ if (Ty->isIntegerTy(8))
+ return ElementCount::getScalable(16);
+ } else if (ISA == VFISAKind::RVVM1 || ISA == VFISAKind::RVVM2 ||
+ ISA == VFISAKind::RVVM4 || ISA == VFISAKind::RVVM8) {
+ // Because 'vscale = VLENB/8', so the ElementCount should be
+ // 'vscale x (LMUL * 64 / sizeof(Type))'.
+ unsigned Number = 1;
+ unsigned LMUL = 1;
+ unsigned ElemCount;
+
+ // TODO: need to distingush rv32 and rv64.
+ if (Ty->isPointerTy())
+ return std::nullopt;
+
+ if (Ty->isIntegerTy(64) || Ty->isDoubleTy())
+ Number = 1;
+ if (Ty->isIntegerTy(32) || Ty->isFloatTy())
+ Number = 2;
+ if (Ty->isIntegerTy(16) || Ty->is16bitFPTy())
+ Number = 4;
+ if (Ty->isIntegerTy(8))
+ Number = 8;
+
+ if (ISA == VFISAKind::RVVM2)
+ LMUL = 2;
+ else if (ISA == VFISAKind::RVVM4)
+ LMUL = 4;
+ else if (ISA == VFISAKind::RVVM8)
+ LMUL = 8;
+
+ ElemCount = LMUL * Number;
+ return ElementCount::getScalable(ElemCount);
+ }
return std::nullopt;
}
diff --git a/llvm/lib/Transforms/Utils/InjectTLIMappings.cpp b/llvm/lib/Transforms/Utils/InjectTLIMappings.cpp
index 92b5d444aac340..8456e4294083bf 100644
--- a/llvm/lib/Transforms/Utils/InjectTLIMappings.cpp
+++ b/llvm/lib/Transforms/Utils/InjectTLIMappings.cpp
@@ -109,11 +109,11 @@ static void addMappingsFromTLI(const TargetLibraryInfo &TLI, CallInst &CI) {
TLI.getWidestVF(ScalarName, WidestFixedVF, WidestScalableVF);
for (bool Predicated : {false, true}) {
- for (ElementCount VF = ElementCount::getFixed(2);
+ for (ElementCount VF = ElementCount::getFixed(1);
ElementCount::isKnownLE(VF, WidestFixedVF); VF *= 2)
AddVariantDecl(VF, Predicated);
- for (ElementCount VF = ElementCount::getScalable(2);
+ for (ElementCount VF = ElementCount::getScalable(1);
ElementCount::isKnownLE(VF, WidestScalableVF); VF *= 2)
AddVariantDecl(VF, Predicated);
}
diff --git a/llvm/test/CodeGen/Generic/replace-intrinsics-with-veclib.ll b/llvm/test/CodeGen/Generic/replace-intrinsics-with-veclib.ll
index fde6cb788b46f9..1bd929f836f9ed 100644
--- a/ll...
[truncated]
|
1. Rename LIBMVEC_X86 into LIBMVEC to support libmvec in risc-v.
2. Add RVVM1/2/4/8 in VFISAKind to distingusih the LMUL value.
3. Declare some RVV vector math functions in VecFuncs.def.
In VecFuncs.def, I add the LI_DEFINE_VECFUNC of LIBMVEC_RVV as follow:
```
TLI_DEFINE_VECFUNC("sin", "_ZGV1Nxv_sin", SCALABLE(1), "_ZGVr1Nxv")
TLI_DEFINE_VECFUNC("sin", "_ZGV2Nxv_sin", SCALABLE(2), "_ZGVr2Nxv")
TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV1Nxv_expf", SCALABLE(2), "_ZGVr1Nxv")
TLI_DEFINE_VECFUNC("llvm.exp.f32", "_ZGV2Nxv_expf", SCALABLE(4), "_ZGVr2Nxv")
```
The `VEC` of TLI_DEFINE_VECFUNC (e.g., `_ZGV2Nxv_sin`), its name mangling rules
defined in
riscv-non-isa/riscv-elf-psabi-doc#455
Now it's still under review.
The `VF` (e.g., `SCALABLE(2)`), it should be `vscale x (LMUL * 64 / sizeof(Type)`.
The `VABI_PREFIX` (e.g., `_ZGVr1Nxv`), `r` means RISC-V vector extension, `1` is
the LMUL value.
```
_ZGVr1Nxv --> RISC-V Vector Extension with LMUL=1
_ZGVr2Nxv --> RISC-V Vector Extension with LMUL=2
_ZGVr4Nxv --> RISC-V Vector Extension with LMUL=4
_ZGVr8Nxv --> RISC-V Vector Extension with LMUL=8
```
34679d6 to
023a029
Compare
|
@mga-sc Hi, can you help to review the pr? I add RVVM1/2/4/8 in VFISAKind to support multiple LMUL value, maybe it can take full advantage of rvv register group. Your pr (#114014) looks like just support LMUL=2. |
mag-mga
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Finally, libmvec is still not ported to RISC-V (today check glibc ToT with --enable-mathvec). I think we should wait for this and also wait for commit to riscv-elf-psabi-doc. Probably, implementation will differ from the proposals.
|
|
||
| /* Verify no warning when math-errno is re-enabled for a different veclib (that does not imply -fno-math-errno). */ | ||
| // RUN: %clang -### --target=aarch64-linux-gnu -fveclib=ArmPL -fmath-errno -fveclib=LIBMVEC %s 2>&1 | FileCheck --check-prefix=CHECK-REPEAT-VECLIB %s | ||
| // RUN: %clang -### --target=aarch64-linux-gnu -fveclib=ArmPL -fmath-errno -fveclib=LIBMVEC-X86 %s 2>&1 | FileCheck --check-prefix=CHECK-REPEAT-VECLIB %s |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why did you leave x86 for arm here?
| #elif defined(TLI_DEFINE_LIBMVEC_RVV_VECFUNCS) | ||
| // GLIBC Vector math Functions for RISC-V | ||
|
|
||
| TLI_DEFINE_VECFUNC("sin", "_ZGV1Nxv_sin", SCALABLE(1), "_ZGVr1Nxv") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You have missed r letter in mangled vector name`after ZGV according to mention MR
| RVVM1, // RISC-V Vector Extension LMUL=1 | ||
| RVVM2, // RISC-V Vector Extension LMUL=2 | ||
| RVVM4, // RISC-V Vector Extension LMUL=4 | ||
| RVVM8, // RISC-V Vector Extension LMUL=8 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are you sure we need to enable all of them (now enabled only one for each RISC-V and AArch64)?
| TLI.getWidestVF(ScalarName, WidestFixedVF, WidestScalableVF); | ||
|
|
||
| for (bool Predicated : {false, true}) { | ||
| for (ElementCount VF = ElementCount::getFixed(2); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Leave a comment about this change. It is unclear why this change is needed and whether it affects other vector libraries.
In VecFuncs.def, I add the LI_DEFINE_VECFUNC of LIBMVEC_RVV as follow:
The
VECof TLI_DEFINE_VECFUNC (e.g.,_ZGV2Nxv_sin), its name mangling rules defined in riscv-non-isa/riscv-elf-psabi-doc#455 . Now it's still under review.The
VF(e.g.,SCALABLE(2)) should bevscale x (LMUL * 64 / sizeof(Type). For example, the type of the parameter of sin is double, so for_ZGV2Nxv_sin, its VF isvscale x 2.In the
VABI_PREFIX(e.g.,_ZGVr1Nxv),rmeans RISC-V vector extension,1is LMUL value.